Sea-Surface Object Detection Based on Electro-Optical Sensors: A Review

Sea-surface object detection is critical for navigation safety of autonomous ships. Electrooptical (EO) sensors, such as video cameras, complement radar on board in detecting small obstacle sea-surface objects. Traditionally, researchers have used horizon detection, background subtraction, and foreground segmentation techniques to detect sea-surface objects. Recently, deep learning-based object detection technologies have been gradually applied to sea-surface object detection. This article demonstrates a comprehensive overview of sea-surface object-detection approaches where the advantages and drawbacks of each technique are compared, covering four essential aspects: EO sensors and image types, traditional object-detection methods, deep learning methods, and maritime datasets collection. In particular, sea-surface object detections based on deep learning methods are thoroughly analyzed and compared with highly influential public datasets introduced as benchmarks to verify the effectiveness of these approaches. The arti

Fatigue Detection for Ship OOWs Based on Input Data Features, from The Perspective of Comparison with Vehicle Drivers: A Review

Ninety percent of the world’s cargo is transported by sea, and the fatigue of ship officers of the watch (OOWs) contributes significantly to maritime accidents. The fatigue detection of ship OOWs is more difficult than that of vehicles drivers owing to an increase in the automation degree. In this study, research progress pertaining to fatigue detection in OOWs is comprehensively analysed based on a comparison with that in vehicle drivers. Fatigue detection techniques for OOWs are organised based on input sources, which include the physiological/behavioural features of OOWs, vehicle/ship features, and their comprehensive features. Prerequisites for detecting fatigue in OOWs are summarised. Subsequently, various input features applicable and existing applications to the fatigue detection of OOWs are proposed, and their limitations are analysed. The results show that the reliability of the acquired feature data is insufficient for detecting fatigue in OOWs, as well as a non-negligible invasive effect on OOWs. Hence, low-invasive physiological information pertaining to the OOWs, behaviour videos, and multisource feature data of ship characteristics should be used as inputs in future studies to realise quantitative, accurate, and real-time fatigue detections in OOWs on actual ships

Multi-Scale Object Detection Model for Autonomous Ship Navigation in Maritime Environment

Accurate detection of sea-surface objects is vital for the safe navigation of autonomous ships. With the continuous development of artificial intelligence, electro-optical (EO) sensors such as video cameras are used to supplement marine radar to improve the detection of objects that produce weak radar signals and small sizes. In this study, we propose an enhanced convolutional neural network (CNN) named VarifocalNet * that improves object detection in harsh maritime environments. Specifically, the feature representation and learning ability of the VarifocalNet model are improved by using a deformable convolution module, redesigning the loss function, introducing a soft non-maximum suppression algorithm, and incorporating multi-scale prediction methods. These strategies improve the accuracy and reliability of our CNN-based detection results under complex sea conditions, such as in turbulent waves, sea fog, and water reflection. Experimental results under different maritime conditions show that our method significantly outperforms similar methods (such as SSD, YOLOv3, RetinaNet, Faster R-CNN, Cascade R-CNN) in terms of the detection accuracy and robustness for small objects. The maritime obstacle detection results were obtained under harsh imaging conditions to demonstrate the performance of our network model

Fast Path Planning for Autonomous Ships in Restricted Waters

Presently, there is increasing interest in autonomous ships to reduce human errors and support intelligent navigation, where automatic collision avoidance and path planning is a key problem, especially in restricted waters. To solve this problem, a path-guided hybrid artificial potential field (PGHAPF) method is first proposed in this paper. It is essentially a reactive path-planning algorithm that provides fast feedback in a changeable environment, including dynamic target ships (TSs) and static obstacles, for steering an autonomous ship safely. The proposed strategy, which is a fusion of the potential field and gradient methods, consists of potential-based path planning for arbitrary static obstacles, gradient-based decision-making for dynamic TSs, and their combination with consideration of the prior path and waypoint selection optimization. A three-degree-of-freedom dynamic model of a Mariner class vessel and a low-level controller have been incorporated together in this method to ensure that the vessel’s positions are updated at each time step in order to acquire a more applicable and reliable trajectory. Simulations show that the PGHAPF method has the potential to rapidly generate adaptive, collision-free and International Regulations for Preventing Collisions at Sea (COLREGS)-constrained trajectories in restricted waters by deterministic calculations. Furthermore, this method has the potential to perform path planning on an electronic chart platform and to overcome some drawbacks of traditional artificial potential field (APF) methods

An Efficient Ship Automatic Collision Avoidance Method Based on Modified Artificial Potential Field

A novel collision avoidance (CA) algorithm was proposed based on the modified artificial potential field (APF) method, to construct a practical ship automatic CA system. Considering the constraints of both the International Regulations for Preventing Collisions at Sea (COLREGS) and the motion characteristics of the ship, the multi-ship CA algorithm was realized by modifying the repulsive force model in the APF method. Furthermore, the distance from the closest point of approach-time to the closest point of approach (DCPA-TCPA) criterion was selected as the unique adjustable parameter from the perspective of navigation practice. Collaborative CA experiments were designed and conducted to validate the proposed algorithm. The results of the experiments revealed that the actual DCPA and TCPA agree well with the parameter setup that keeps the ship at a safe distance from other ships in complex encountering situations. Consequently, the algorithm proposed in this study can achieve efficient automatic CA with minimal parameter settings. Moreover, the navigators can easily accept and comprehend the adjustable parameters, enabling the algorithm to satisfy the demand of the engineering applications

An Efficient Ship Automatic Collision Avoidance Method Based on Modified Artificial Potential Field

A novel collision avoidance (CA) algorithm was proposed based on the modified artificial potential field (APF) method, to construct a practical ship automatic CA system. Considering the constraints of both the International Regulations for Preventing Collisions at Sea (COLREGS) and the motion characteristics of the ship, the multi-ship CA algorithm was realized by modifying the repulsive force model in the APF method. Furthermore, the distance from the closest point of approach-time to the closest point of approach (DCPA-TCPA) criterion was selected as the unique adjustable parameter from the perspective of navigation practice. Collaborative CA experiments were designed and conducted to validate the proposed algorithm. The results of the experiments revealed that the actual DCPA and TCPA agree well with the parameter setup that keeps the ship at a safe distance from other ships in complex encountering situations. Consequently, the algorithm proposed in this study can achieve efficient automatic CA with minimal parameter settings. Moreover, the navigators can easily accept and comprehend the adjustable parameters, enabling the algorithm to satisfy the demand of the engineering applications

A Practical Risk-Based Model for Early Warning of Seafarer Errors Using Integrated Bayesian Network and SPAR-H

Unsafe crew acts (UCAs) related to human errors are the main contributors to maritime accidents. The prediction of unsafe crew acts will provide an early warning for maritime accidents, which is significant to shipping companies. However, there exist gaps between the prediction models developed by researchers and those adopted by practitioners in human risk analysis (HRA) of the maritime industry. In addition, most research regarding human factors of maritime safety has concentrated on hazard identification or accident analysis, but not on early warning of UCAs. This paper proposes a Bayesian network (BN) version of the Standardized Plant Analysis Risk–Human Reliability Analysis (SPAR-H) method to predict the probability of seafarers’ unsafe acts. After the identification of performance-shaping factors (PSFs) that influence seafarers’ unsafe acts during navigation, the developed prediction model, which integrates the practicability of SPAR-H and the forward and backward inference functions of BN, is adopted to evaluate the probabilistic risk of unsafe acts and PSFs. The model can also be used when the available information is insufficient. Case studies demonstrate the practicability of the model in quantitatively predicting unsafe crew acts. The method allows evaluating whether a seafarer is capable of fulfilling their responsibility and providing an early warning for decision-makers, thereby avoiding human errors and sequentially preventing maritime accidents. The method can also be considered as a starting point for applying the efforts of HRA researchers to the real world for practitioners

Multi-Scale Object Detection Model for Autonomous Ship Navigation in Maritime Environment

Accurate detection of sea-surface objects is vital for the safe navigation of autonomous ships. With the continuous development of artificial intelligence, electro-optical (EO) sensors such as video cameras are used to supplement marine radar to improve the detection of objects that produce weak radar signals and small sizes. In this study, we propose an enhanced convolutional neural network (CNN) named VarifocalNet * that improves object detection in harsh maritime environments. Specifically, the feature representation and learning ability of the VarifocalNet model are improved by using a deformable convolution module, redesigning the loss function, introducing a soft non-maximum suppression algorithm, and incorporating multi-scale prediction methods. These strategies improve the accuracy and reliability of our CNN-based detection results under complex sea conditions, such as in turbulent waves, sea fog, and water reflection. Experimental results under different maritime conditions show that our method significantly outperforms similar methods (such as SSD, YOLOv3, RetinaNet, Faster R-CNN, Cascade R-CNN) in terms of the detection accuracy and robustness for small objects. The maritime obstacle detection results were obtained under harsh imaging conditions to demonstrate the performance of our network model